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1 Introduction 



In the history of computational linguistics* the first significant question asked was how to 
analyze syntax Programs that were developed using only syntactic analysis were unsuccessful at 
actually understanding natural language and consequently the emphasis shifted to semantics. 
Subsequent programs using piimanly j^emantic methods met with a beginning level of success. 
Since Winograd [221 m an effort to further refine this success, the basic question guiding artificial 
intelligence work has been shaipen» tl to the question of how to best represent semantic information 
(this has been called "the represrnt uion problem") (I) In its extreme form, the current emphasis 
on semantics excludes any conM^ln :^tion of syntax. A typica. view is that of Schank [171 who 
claims about his system that "syntax has been denigrated to the status of something to use when alt 
else fails" 

Most of the recent discussion concentrates on the holistic u^ture of language and, rather than 
denying syntax a role, stressrs thr inreraction of syntactic and nantic features of language and 
cautions ugainst the possibility of making hard and fast divisions into "syntactic" and "semantic" 
problems. In actual programs, this view is implemented by ^ elimination of distinct phases of 
analysis corresponding to syntax ind semantics. Instead both syntactic and semantic procedures are 
available which can call each othrr, and the results of each call determine how the analysis will 
proceed. Thus the result of both the extreme and the more common view of the relation between 
these two components of langu^re is the exclusion of any serious discussion of the general 
systematic role that syntax plays in language understanding An even more disturbing trend that 
has appeared m recent discussions is the assumption that no logically coherent theory of meaning 
for natural language is even possible (2) 

Certainly the early failures in mechanical translation and other types of purely syntactic 
analysis have shown that the traditional conception of syntax is weak; and efforts to use such 
logical techniques as resolution thrnirm proving have shown them to be inadequate for solving the 
problems of natural language mfrimce These failures are far enough behind us that we can 
perhaps restore some balance into the discussion. 

We make two claims: 

(I) Syntax is important to the understanding of natural language. 



(1) The "semantic information" question is currently being phrased as a debate between 
procedural and declarative semantics 

(2) See for example Colby and Enea [2], Wilks [20]. 
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(2) A logically coherent theory of meaning is not only possibie. but necessary if 
computationa! linguistics is to proceed 



We shall, m the body of ih'» paper, elaborate these claims, present a computer system 
(CONSTRUCT) that was desitii'''^l to support these claims, and then suggest what kind of 
mathematical analysis is necessaiy to make progress in the area 



l.i The Role of Syntax 

The "denigration of svnt.i\'* ^e^ms to be associated with the belief that ordinary language is 
sufficiently less orderly and "nilf likp {}■) than is required for formal analysis. On the other hand. 
Richard Montague [9] has supv'/'^?»\1 that there is no important theoretical difference between 
formal and natural languaf"^<. 1>. .pirr thr popularity of the informal approach, we think that 
Montague's position will be vindn 

What IS needed is a richer view of syntax that accounts for the appearance of disorder. We 
suggest that the correct analvMs uf the syntax of a language provides a method of discovering the 
computational control structure i.t the sentences in the language -- to use computer metaphors, the 
"program" that must be "execuipd " in order to answer a question, execute a command, or evaluate 
the truth value of a declarative In order to answer the natural language questions asked of it. the 
computer must execute a prov;rani to compute the answer. Thus, the "semantics" of a sentence is a 
program generated by the "synt 4\ " 

The execution of such a "provram" will in general involve steps that modify the program 
itself. These steps correspond to plares where the syntax fails to be as orderly as first-order logic. 
We describe these situations as u->'iar,tic nansformations. 



1.2 Towards a Theory of Meaning 

The claim that English is ho^t rhoiight of as a "formal language" is motivated by the success 
in model theory, a part of foimal logic Alfred Tarski [19] characterized truth for the formal 
languages by saying that "a true sentence is one which says that the state of affairs is so and so, 
and the state of affairs indeed is so and so". (Pp. 153-155 of [19].) The programme of carrying 
this characterization of meaning through to natural languages involves giving a model structure 
that represents the world under discourse, and showing how to map from sentences (or sequences of 
sentences) to conditions of this model structure. 



(3) See Wilks [20] for a discussion of this view. 
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2 The CONSTRUCT System 



Before we discuss the abovr theories in greater detail, let us describe the CONSTRUCT 
system. CONSTRUCT is .1 tirxible modular system for natural language processing. The 
CONSTRUCT program itself which is written in SAIL serves as the executive component of the 
system. It contains a built-in scanner and parser as well as all the routines for interfacing and 
coordinating the other modulp-^ ot the system and also provides extensive facilities for on-line 
monitoring of the system Deiicnding on the actual modules used the CONSTRUCT system can 
serve many purposes ana ' handle many diverse fragments of natural language. The 
implementation discussed in this jKiper is a question-answering system for elementary mathematics. 
(For more details on this system st?e N Smith [18].) 

In addition to the built i;i scanner and parser, the current system Is composed of I) a 
dictionary of the words in the vccnbulary and their lexical categories, 2) a special dictionary called 
the TRANSL dictionary of strint;s of words designated for special preprocessing, 3) a context free 
grammar which has 4) a semantic function associated with each rule of the grammar and 5) an 
evaluation program written in USP 



2.1 Preprocessing 

The operation of the ^vstrm can be viewed as a four-stage process. The first stage is 
preprocessing which is conducttHi iindei the direction of the scanner. CONSTRUCT's scanner is 
similar to scanners found in coni(.'il»?i? It uses a table of break characters to separate the input into 
words Next certain strings of woid<; ate eliminated or replaced. The TRANSL file contains a 
complete list of the strings to bo Mib<:iituted for and the substitution strings. Currently four types 
of substitutions are made 1) ahhioviations are TRANSL'd to the full word form, 2) synonyms are 
TRANSL'd to the most coniinonlv used one of the group, ?) common phrases which have a 
meaning not strictly determined by the combination of the individual words are TRANSL'd to a 
single word representation, and 4) noise words are eliminated. 

One important function of the TRANSL component is to provide pattern-recognltion 
capabilities. Colby and Enea [?.] use question-introducers as an example to explain their pattern 
recognition technique. Certain polite phrases that cannot be analyzed literally are used to introduce 
questions. Essentially the problem is to fill a syntactic slot with an unanalyzed phrase. Their rules 
for this are: 



. ■ • or SENTENCE • 

QUEST i ON- lNTRODUfER>: Q cNOUN-PHRASe>:N 

^ "S !N '«•?'* ): 
•a.cS OF QUESTION-lNTRODUr 
COULD YGU TELL riE 
UOULO YOU TELL riE 
PLEASE TF L riE 

Our system handles quesiion sn'rodiicers by including each such phrase In the Ti: - l-fSL and 
processing the questions with .ne gi ammatical rules: 

RULE It Q - QUt-STlON-INrROOUCER NP 
EXl: Do gou Know the sum of 2 and 4? 

RULE2: Q QUESTION-INTRCOUCER INTERROGATIVE NP LlNKVERB 
EX2: Do you knou nhat the factors of 12 are? 

It is interesting to note that then system which is designed with pattern recognition as the 
paramount concern requires as many rules to deal with the question-introducer m ours does. We 
need one line in the TRANSL for each of the phrases as they need one rule for each, and we also 
need one grammatical rule for rach sufficiently different construction of the question following the 
phrase Our RULE I is u-alogous to their rule and they would need to add RULE2 before they 
could handle £X2. 

Use of a separate component fen pattern recognition has four definite advantages. The first 
advantage is the speed and pftirifncy gained by an initial preprocessing phase which makes 
possible a shorter grammar A ^rcond advantage to a separate module containing all the strings is 
the ease with which changes cnii i>e made to the strings and to the grammar itself. The third 
advantage Is in the flexibility r.f the system To change subject areas we can use the same basic 
erammar with a different TRAN.SL file while their grammar contains many rules dependent on 
the particular vocabulary of the subjoct area involved. Finally, th; preprocessing approach has the 
advantage of clarity. The output of the scanner shows prccisel> which substitutions were made and 
how the input was standardized In a typical pattern-recognition grammar, this Information Is 
burled In the parse tree 

After all of these substitutions have been made, the scanner uses the dictionary to form a 
new representation of the input by replacing each word by its lexical category. It Is this 
representation which is passed to the parser. 



o 

ERIC 



2.2 Syntactic and Semantic Analysis 



The second and third Maf;r«i of processing proceed in parallel. The second stage is the 
syntactic parse of the input The third stage is the creation of what we call a 'semantic 
construction' for the input. It hr^ the form of a LISP S-expression which is ready for evaluation. 
Each rule of the grammar has sn associated semantic function whose explicit arguments are the 
meanings of the elements on the right hand-side of the rule. The evaluation of this function yields 
the meaning of the element on the left-hand-side of the rule. As the syntactic parse proceeds the 
"semantic parse" builds up the final semantic construction for the input from the semantic functions 
associated with each rule used in the syntactic parse. 

Thus there is no interaction between "syntactic" and "semantic" routines at runtime. Many 
systems use an interactive process in which parsing begins but semantic routines can be called in to 
disambiguate certain phrases and the semantic routines in turn can call for further parsing of a 
piece of the input, etc Instead of an interactive process of this type, our grammar is designed to 
obtain the semantic information from the syntactic parse. This Is based on the assumption that a 
sentence contains two types of information The first type of information is carried by the 
substantive or content words in the sentence These words indicate which items in the data base or 
world model used by the evaluation program are to be dealt with in evaluating this particular 
sentence The second type of information which can be obtained from the sentence Is carried by 
both the functional words used in the sentence and the organization or grammatical structure of 
the sentence. This information tell^ us which of the evaluation procedures available are to be 
applied to the relevant item? in !h^^ data base and also gives us the control structure to use m 
applying these functions or piofrdnros. 

Our grammar has been wiiitti) to maximize the amount of this type of semantic information 
that can be obtained from the syntactic parse. The grammar is quite different from the usual 
grammars u.«ed for natural lan^uat^e processing which traditionally start with the rule 

S - NP VP . 

It would be virtually impossible to construct a semantic function to ?ttach to this rule. Not only is 
this rule too general but it has already begun an incorrect breakdown of the sentence for semantic 
purposes From this point on in the |)arse. the mam noun phrase is separated from both the verb 
which will indicate what action to take with the noun phrase and from any clause in the VP 
following the verb. Another exam|)le of a series of traditional rules which lead to a parse which 
would have to be judged a poor parse on semantic grounds is 



NP DETERHINER N 
NP -» NP PREPHRASE 
NP ^ N 

PREPHRASE - PREP NP 
OETERniNER -» ARTICLE 



Consider the phrase "the factors of 4". which would be parsed by these rules as shown in Figure I. 




NP PREPHRASE 

/\ /\ 

DET N PREP NP 

I \ II 

ARTICLE factors of N 

I 



the 



4 



Fitiure 1 Tree for the factors of 4. 

The information as to uluch typ*? of determiner is used is ntt discovered urtil the lowest 
level in the parse and even th. n bcth the definite and indefinite article are lumped into one lexical 
category. This has no correspondf nee to the actual scope of the determiner and allows no point in 
the parse for assigning the pariicuMi function associated with each quantifier and article. In our 
system, each semantic function (exirpt for the explicitly transformational ones, see Section 4 below) 
is assigned at a level where all the ^^guments to the function are available. 

These rules create a simil.ii problem with the preposition. Traditionally all prepositions 
have been grouped into one l«?xi' al category and parsed by the same grammatical rules. The 
semantic insight about "of" which is used by our system is that this preposition is used to designate 
a function, e.g.. the FACTOR function and its argument($), e.g., 4. Other examples arc: the 
denominator of 1/2, the sum of 2 and 4, and the intersection of {a.b} and {a.c}. We use the 
semantic function APP for applying functions in the data base to their specified arguments. The 
traditional rules split the function from its argument in the parse tree and provide no rule with 
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which the APP function can be associated We have assigned "of" its own lexical category and also 
created a lexical category FCN tor nouns which are names of functions Thus the rule is 

NP - FCN \Crs NP (APP ;1; :3;) 

[Note: The form .n. in the specification of the arguments to a semantic 
function refers to the ivh element on the nght-hand-side of the rule. . 
These slots will be filled in by the semantic functions for those elements as 
the parse proceeds ] 

As can be seen in this example, our grammar differs from traditional grammars in two basic 
wavs - first by assigning words "lev,r,r categories on the basis of their semantic role and second 
bv making sure that each rule .n the grammar has semantic relevance. This has the affect of 
considerably flattening the parse t.ees so that more of the context is considered at each level. 
However it should be made clca. thst we are not in the position of needing virtually a separate 
rule for each sentence We do not create categories, either lexical or grammatical, on the basis of a 
priori syntactic considerations f)Ut we have found it possible to create many broad categories on 
>emantic grounds. Consider the sentence types: 

;) Hoxo rnany X are f 
2) Whichxartyf 
9) Is X V? 

4) Give the x that is y' 

We have created the gramm.uc.l category SUBST to cover all the grammatical constructions 
which can fill the y slot The bUS^T rules are: 

Rule Example 

SUBST NP Is 2 a factor of 4? 

SUBST ADJ ^^ou many factors of 12 a""* e^®"? 

SUBST -» ARITHRELS Hhich factor of 12 is greater than b.' 

SIJBST -» PREPP Give the factors of 12 that are between 1 and B! 

This group contains a suipt isingly diverse range of traditional syntactic categories. However 
viewed on semantic grounds th^ir function in the example sentences is identical. Each of the types 
of SUBST's is semantically rcp.esr.uod by a constructive set. Thus we have the set of factors of 4. 
the set of even numbers, the set of numbers greater than 6. and the set of numbers between 1 and 
6 Some of these sets such as the set of even numbers are infinite sets. In constructive set theory 
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.■>,:i«»ie <ets I p|)res*»nfp(l I'V ,< (h u u'tnistic function rather than by a list of memoers, as wi» 
shall describe in Section ? below 

Considering that the v in the sentence types will also be a set, it is easy to see what the rules 
=ind their associated semantic functions should be for the four sentence types given. C stands for 
the cardinality of a set, I for the intersection of two sets, and S for the subset relation on two sets. 

1) Q ^ \HOUnANY\ NP LINK SUfiST (C 'I ;2: t4t)) 

2) Q -» INTERROGATIVE NP LINK SUBST il \2x $4j) 

3) Q - LINK NP SUBST <S i2; ;3;) 

4) C \GIVE\ NP (:2:) 

NP NP RELPRONS (I ;1; ;2;) 

flFLPRONS - RELPRON LINK SUBST {|3;) 



The use of a greater variety ot lexical categories does have the affect of lengthening the 
grammar but it also has advantages Like the DEACON system [3], we use a separate lexical 
category for each function word The reasons fo: this are obvious In a system which aims at 
extracting all the semantic information possible during the parse. However It is not only the 
functional words which have '^een carefully categorized but also the substantive words. For 
example, nouns are currently divided into the two categories N and FCN The FCN's are nouns 
which name functions, eg. factoi. uim. and intersection. The N's are nouns which name sets 
(constructive sets) e.g., fraction, luimber. and prime 

Our handling of the classification of substantive words, which is very similar to thai of 
Sager [16], is based on the premise that categories can be formed which contain words that are 
both naturally related to each other with regard to the subject matter and naturally related with 
regard to their grammatical role In the case of our nouns, this is obvious. The division Into FCN 
and N clearly has semantic relevance It also clearly has grammatical relevance because FCN's as 
function names will always be accompanied by their argument and this argument slot must be 
taken account of in the gramnvir Thus the division into categories adds syntactic precision to the 
grammar as well as increasing us power 3s a tool for extracting semantic information. 

When the parse is complete, the semantic construction which represents the semantic surface 
structure is passed to the evaluation program. Before discussing the evaluator, a comment should 
be made on the ambiguity problem. One motivation for performing syntactic and semantic 
analysis interactively Is so that the semantic routines can be used to resolve syntactic ambiguities as 
they arise. Our system however also provides a natural means of disambiguation. Most of the 
lexical ambiguity which arises from words having multiple lexical categories is resolved by the 
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grammar since generally only oi.»^ of the forms will parse. Grammatical ambiguity which may arise 
from unresolved lexical ambigmty or from ambiguous parsings of a single lexical form is often 
resolved by default in those cases where all the grammatical parses result in the same semantic 
construction. Semantic ambiguity which arises from ambiguous grammatical parses with different 
semantic constructions is resolved l)y the evaluation phase if only one of the semantic constructions 
can be successfully evaluated. Thus the natural progression of the system eliminates spurious 
ambiguities The remaininp ambiguities are those which result when more than one alternative 
semantic construction can be successfully evaluated or when the evaluation of a single semantic 
construction reveals an ambiguity because of multiple senses of an item in the data base. These 
ambiguities will have to be resolved by the evaluator. The evaluator is the natural place for this 
disanibieuation to be performed since it has access to the data base or worSd model including 
information about the context of the conversation and it can be provided with a wide range of 
heuristic disambiguation procedures 

2.S Evaluation 

The evaluator is a LISP ptogr^m which evaluates the semantic constructions using a 
recursive inside-out technique The evaluation is essentially independent of the natural language 
processing components. This has ^rveral advantages. First it can be programmed m a different 
laneuage Second the fact thar thr h.itural language processing routines are not dependent on the 
form of the data base means that D the entire system does not need to be reprogrammed in order 
to take advantage of new ways of lepiesenting knowledge in the data base (which is a rapidly 
advancing field), and b) diffe.enr types of knowledge can be stored m different types of data 
structures Also the various srmmric functions can be programmed using different data base 
manipulation techniques and the code for these functions can easily be changed or extended. 



3 Constructive Semantics 

Bv saving that CONSTRUCT embodies a theory of meaning, we mean that there is a set* 
theoretical characterization of the "world" that the CONSTRUCT fragment Is referring to. and 
that there is a map between the sentences of the fragment and objects in this set-theoretical world. 
For example, consider the sentence 

Hou) many factors of 4 are there that are also multiples of 2f 
The sentence 1$ a question, and the correct answer is the cardinality of a set, i.e., 



I Ix|k is 0 fictor of 41 n Ixlx iS a multiple of 21 | 

The set-theoretical structure that woulo be required to deal with this sentence would have to have 
a FACTOR and MULTIPLE function over the domain of natural numbers, as well as the logical 
and set manipulation functions 

This kind of semantics was developed to deal with the formal languages of mathematical 
logic. Tarski in {\9] showed how to give composition rules for the syntax of first-order logic and 
how to characterize the meaning for the formulae built up from those composition rules. 

The power and importance of the method soon became obvious to logicians. In examining 
the models that a particular form.il language had. it became clear what the expressive flower of 
that language was. The metaphots of mtended interpretation were stripped away, leaving for each 
theory an exact appraisal of what the theory really was. Thus, conflicting accounts of such 
historically difficult problems as entailment, tense, and modality became analyzable in a systematic 
way. 



31 Objections to Modcl-thcnretical Semantics 

There have been senous- Moubrs raised about the relevance of logic in general to 
computational linguistics. Mo.^t of the substantive objections are only about the technique of 
representing information usinf; prrdicate calculus and then using resolution theorem proving to 
provide the inference machinery Minsky [8] raises a somewhat more general objection that 
stresses the holistic character of "common sense" reasoning; he argues that one cannot "confront . . 
[a logical] system with a realistic;illv large set of propositions." (4) Nevertheless, Minsky still 
considers only the proof -theoretic i^pects of mathematical logic, and does not really deal with the 
semantic aspects that we are suggosting are relevant to computational linguistics. 

Objections specifically about model-theoretic semantics include the following: 

1) "While form.illy .adequate, set-theoretical semantics is not the way to 
look at natural language for computers." We have no ready answer to this 
objection except to note that it applies equally well to all other theories. 

2) "Set-theoretical semantics commits one to a referential view of language 
- that words refer to things " But the "objects" in the set structure could be any 
theoretical or abstract construction, just as the "atoms" in a LISP program need 



(4) P. 73 of [8]. 
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not be material ob)^^as Set-theoretical semantics is independent of 
metajDhysical and episteinological commitments. 

3) The notion ot 'defining the meaning' m set-theoretical semantics 
presupposes our ability to perform impossible computations" This belief is 
based on a confusion about what it means to "define" or "evaluate" the meaning 
of a sentence. The sen^e of "evaluation" used by logicians means to^ give a 
characterization of the truth conditions of a sentence. The "evaluation" notion 
current in computer science is that of carrying out a compulation that leads to 
an answer. Knuth [Gj has cnlled these two notions existence and construction. 
!t IS important to note that we can know the truth conditions (i.e., the 
"meaning") without bPUK; able to carry out the computations those conditions 
imply. Patrick Suppes' example is the question: 

Are there natural numbers x,y,z bigger than 0. n bigger than 2, such that 

It IS perfectly reason iblr tn say that we know what this question "means" - it is 
a statement of Fermat's last theorem - but that we cannot answer it yet. 



3.2 Computer Implcmrnt.it inn 

This last objection is quite serious in that any computer implementation of a set-theoretical 
semantics must contend with intmiie set structures in a computationally graceful way. For example 
the question 

Hciii many even numbers are prime? 

involves three infinite set in its meaning conditions: 

I {x|x is event n Ixjx is a number) n Ix|k is primel | 

The theoretical solution is to base the semantics on constructive set theory, in which potentially 
infinite (or very large) sets are represented by characteristic functions. These functions are then 
available to be called directly, combined with other functions, or examined for information 
extraction. The computations! pinblems are then no different than in any similar system, except 
that one has an exact charactetization of the conditions of success. (5) 

(5) See Rawson [15] for a description of the feature extraction and heuristic techniques that 
were used in the evaluator to com|nite the answers to questions like the above. 



II 



12 A Semantic Theory Tor Computational Linguistics 

We propose constructive u^mcntu^s as the basis for computational linguistics. Of course this 
siemmtics is particularly suited fo rhe fragment that we chose (elementary mathematics). But the 
importance of constructive srm tiKKS « xrends to other systems with a less exact subject matter, and 
there are benefits to be obtamrd from the application For example, we are rather impressed with 
the notions of inference of Rog> r Sch^nk and his associates [17] We think that their ideas would 
improve by having their conceptual dependency system formalized with a modal logic semantics. 



4 Semantic Transforiuatinns 

A common theme m ctMnpiitationa! linguistics is to show how a given system obtains 
"transformaiional" powers WockI^ in [2?] showed how his augmented state transition networks 
extend beyond the powei of ronroxt tree grammars to cover syntactic transformations. Wmograd 
[22] argued that his procediirr^ obr.^m similar capabilities by simply being programmed to do so. 
Petnck [13] used an explicitly n an^foimational system. 

Beginning with Chomsky [1] p^enerative grammarians postulated a syntactic deep structure 
with transformations mapping to the surface structure. One motivation for the appeal to the 
transformation was that the un^Klfvl ronte\t-free grammar was clearly insufficient to account tor all 
the syntactic features observed in natural language 

This point was made quite forcefully by Postal in [14], Using the embedding properties ot 
context-free languages, Postal demonstrated that certain constructions in Mohawk could not be 
recognized using any context fire grammar. The importance of his argument rests on the 
unquestioned assumption that the grammar should account for all "syntactic" features (such as 
inflection) and provide a "structural description" that accounts for "all the grammatical information 
about a sentence which is in principle available to the native speaker". (6) 

There is no reason why the lole given to a phrase-structure grammar should be the role that 
Postal assumes. Moreover, the notion of "syntax" that Postal is looking for may be incoherent. 
One of the major theoretical insights of Winograd [22] is that the division into "syntax", 
"semantics", and "pragmatics" may not represent the way that our knowledge is used in 



(6) [141 p. 137. 
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understanding natural language (7) One can give plausible interpretations to such Chomskian [I) 
non>sense sentences and non sentences as "Colorless green ideas sleep furiously." and "Furiously 
sleep Ideas green colorless * (8) 

4.1 Control Structure and Semantic Traiisformatioiis 

The CONSTRUCT system uses a context-free grammar to lay out the surface control 
structure of the semantic functions Some of these functions (such as V LUS) perform a simple 
computation. Other functions, however, cause an interrupt that changes the control structure. 
These semantic functions are transfotmational in that their combined effect is to set up an alternate 
control structure that calls only iunrtions that compute values in the straightforward way. 

Consider the question: 
(I) h 2 or 3 odd'' 

The CONSTRUCT grammar parses (1) with a tree structurally similar to the one in Figure 2: 




V NP ^(*) ADJ 




Is NP or NP odd 



Figure % Surface Syntax Structure of Is 2 or S odd? 



(7) An example that will illustrate this point is "The city councilmen denied the women the 
permit because they were revolutionaries", where determination of the antecedent of they depends 
on knowledge about the political character of city councils. 

(8) See for example Minsky's frames paper [8], pp. 25-26. 



13 



Note that the noun phrase "2 or ^ is parsed at a low level in the grammar, while the semantic 
effect of the word "^or** has its impitt at a higher level where the phrase "odd" is available. Thus, 
the semantic function associated with the node (:<) causes the alternative control structure shown in 
Figure 3 to be set up. 




V NP ADJ V NP AOJ 
Figure ? Semantic Deep Structure of Is 2 or S oddJ 

The structure in Figure ? is the u^mayxtic deep structure for (1) The function associated with (:0 
causes a semantic transformation, 

Winograd*s solution is to mtci i upt the parsing process itself with "demons" to handle special 
words such as ''or*'. The major rli5;acW3ntage of Winograd's method is that the description of the 
flow of control becomes very difficult, and the tendency develops to rely on the code itself as the 
best (if not the only) description of what is happening. (9) 

Another example of a semantic transformation is sentence (2). 

(2) Does 12 have 12 as a factor and as a multiple? 

Figure 4 gives the surface structure, and Figure 5 gives the semantic deep structure obtained after a 
transformational semantic function alters the control structure. 



(9) This reliance on the code can of course be carried to the extreme of suggesting that the 
program itself is a theory of me.ining 
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AUX 

A 

Does 



NP 
I 

12 



V 

I 

have 



HNP_ 



NP HNPADP 

12 / and 

/ 

as a factor as a multiple 

Figure 4. Surface Syntax Smictutt» of l)oti 12 have 12 as a factor and as a multiple? 




a file tor of 12 



a multiple of 12 



Figure 5. Semantic Dpp|> Sti nctiire of Does 12 have 12 as a factor and as a multiple? 

Semantic transformations hip Mdiilai ly useful in handling quantifiers. In a sentence such as 
(4) Does 12 have any factors that <jre even? 

there is a reasonable case to be insdc that rhe "syntactic scope" of the quantifier any extends only to 
the noun phrase in which it occui? The natural "logical scope" of any would however contain 
more of the sentence, extending »ivi i thp predicate phrase. In the CONSTRUCT system, the parse 
provided by the context-free f-iai iinnt associates the quantifier with the noun phrase; then, tht 
evaluation of the semantic funaions associated with these rules sets up the control-structure where 
the KOpe of the quantifier conp^|ionds to its logical scope. 
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4.2 Context-free Cramiii.iis Revisited 

As we suggested previously, .1 t.ood deal of effort has gone into exposing the limitations of 
context-free grammars. It is our view that syntax analysis should only be expected to lay out the 
control-structure of the semantic functions, where some of the functions may be transformational 
(when they correspond to "natural" transformations). This necessitates a rethinking of the 
limitations of any syntax-analysis method. 

For example, one limitation of a context-free grammar is that there cannot be arbitrarily 
many constituents at a given If^vrl This typically happens for adjective phrases, as shown in 
Figure 7. The best that a cont^^xr tipe grammar can do is to use some recursion generating a tree 
as in Figure 8, where the constituents become embedded. 



ADJP 




ADJl 



ADJ2 A0J3 



AOJn 



Figure 6. Immediate Constituents, 
ADJl ADJP 




ADJ2 



ADJP 



\ 

ADJn 



Figure 7. Contpxt froe "approximation" of Immediate Constituents. 



The real difficulty with FiF;uie is that it makes the semantic analysis unwieldy. In the 
examples in the CONSTRUCT fragment, the typical situation is that each ADJi is a set structure, 
and it Is necessary to have all the sets available at the same level of the semantic analysis in order 
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ro evaluate them correctly In CONSTRUCT, a tranformational semantic function provides Figure 
6 as the semantic deep structurf 

By Mmilar means it is posMhle to show that none of the hmitations ascribed to coniext-frec 
grammars apply when one has semantic functions associated with the rules. 



5 Alternate Approaches to Semantic Transformations 

Several approaches that ^lo tjiiite close in spirit to the semantic transformations of the 
CONSTRUCT system deserve to mentioned. Two such approachs are the interrupts in the 
parsing process of Winograd's hlocks world [221 and the augmented transition networks of Woods* 
system [23]. Two other approaches that are closer to the semantic transformation notion of 
CONSTRUCT are those of Montaiiup and Knuth, 

5.1 Montague and English As A Formal Language 

Richard Montague [9].[K'], (and philosophers such as Moravcsik and Gabbay [11] who have 
proceeded from where Montaguo l»'tf off) an? motivited primarily by the analogy that English is a 
formal language, i e.. simshr lu :h<' hnguiges of fir;;; otdei logic Montague and his co workers 
also noted the above problem of ihf scope of quantifiers Their solution has been to postulate 
larc^e set structures to serve as drnotanons for quantified phrases such as every man and some 
loornan. Then, the meaning of a sentence such as "every man loves some woman" can be defined in 
a way that appears to avoid nny transformational mechanism. Th2 Montague approach is quite 
elegant logically. It fails to satisfy linguists in that their need is for something like the insight into 
psychological reality and srructur<=* of the brain that they believe tranformations provide. (10) For 
computational linguistics, the Montague method is computationally awkward since it escalates the 
complexity of the type structuie bcvond what can be reasonably represented in a machine. One 
way to characterize this escalation ot type structure is to say that information is being encoded at a 
low level (m the denotation of rhe noun phrase), and then being decoded at a higher level (in the 
whole sentence), We feel thai tho theoretical approach of the CONSTRUCT system is more 
computationally feasible In ^^ddltlon it fits into the linguistic tradition of transformational 
grammar, and points the way to eliminating the notion of syntactic deep structure. Fillmore [4] has 
presaged the elimination of syntactic deep structure, calling it "a level the properties of which have 



(10) See Partee [12], p. 24? for a discussion of the relation between Montague grammar and 
traditional transformational grammar 
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more to do with the methodological commitments o^ grammarians than with the nature of hunr^an 
languages'*. 

5.2 Knuth and the SpiiMiitics of Context-free Languages 

Donald Knuth [6] <yU^|.,^'^tO'i :\ nstem of attributes for the nodes of a context-free parse tree. 
The calculation of the values io\ Mi*^*' .attributes proceeds in such a way that information is passed 
both up and down the pai5^^ urr The effects that one obtains are formally equivalent to our 
semantic transformations .^nd to tue Montague set structure encodings of information. The 
approach IS particularly applicjhN the implementation independent definition of programrping 
languages, called "declarative" by Knuth and his students 

Wllner [21] uses a mnlii ssing environment to ci^lculate the values for the Knuth 
attributes of a parse tree for 3 loinpilei called SIMULA 67. He shows that one can use the order 
m which these values are calcuLirrM lo cleterniine an augmenud tierivation tree. (II) The augmented 
derivation tree is quite 5imil.ii to tho semantic deep structures that the CONSTRUCT system 
obtains. One difference is ihar the augmented derivation tree structure arises implicitly from the 
order m which the Knuth attribute's ire assigned values; thus the augmented derivation structures 
are somewhat arbitrarily determiurd by the attributes selected. In the CONSTRUCT system, the 
explicit notion is that we are comjuiting an alternative control structure that arises from the initial 
syntax analysis. 



6 Scheinatology and Transformational Settiantics 

Although we have desciil?rri the theory of language that is embodied in CONSTRUCT in 
terms of model theory and tornnl >*»mantics, the program itself is procedural in nature. For 
example, the evaluation coni|»uti^iir mI ihe system performs dynamic manipulation of functions at 
runtime by creating and destrf»vini. vprcLilized functions as it evaluates the semantic parses passed 
to it. Moreover, the semantic n anMoimations are functions mapping semantic parses to semantic 
parses, and as such, can be descnhf d only in procedural terms. At present, most representations of 
procedural knowledge are in terms of programs, and therefore are not very accessible to the kind of 
model theoretic analysis presented in the preceding sections. A method of analysts of procedural 
knowledge called schematology has been developed that has promise of giving a mathematical 



(I I) Pp. I7M84 of t2ll 
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account of the notion of semantic tiansformation, (12) 



6.1 The Motivation for .iiul ilie Development of Schematology 

Schematology is dPMgnrd m r vnact from a program the control structure of that program. It 
isolates those parts of the pio.. »<n Mich .is conditionals, branches, leturns. and subroutine calls that 
affect the sequence of machmr m'nucaons executed This is done by replacing the actual logical 
conditions and arithmetic i^nd I.m. <1 functions of a program, such as an ALGOL prograni. by 
predicate and function symboh I lu^ is stiictly analogous to the replacement of the actual words of 
a sentence when represcnti.K In^ir.il structure of the sentence usmg first-order logic. By 
studying such abstractions one i,^ .1.1'^ to analyze many programs at once in much the same way 
that first order logic analyzes m niy s^'iitcnces at once 

Much of the work on srhrnvntolo^y has a motivation which is surprisingly similar to a 
oroblem that has plagued linguists wer since Chomsky introduced the notion of a transformation 
in the I950'S"that of finding "n.itui.il" constraints on transformations. Although it is intuitively 
obvious that programming lanr,ii.iye^ vary greatly m their expressive power, it is possible in most 
laneuages to write for any given iransive function a program that will compute that function, and 
accordine to Church's thesis ii is Mr.|.nssible to do any better than that. Hence, most programming, 
laneuaees are universal m tlic '•nur that one can write a program for an arbitrary recursive 
function in them For example, \v,ih ALGOL and machine language are universal m this sense 
despite the fact that ALGOL is d.aily a more powerful and more expressive language than 
machine code 

Given this universality, it is imposMble to compare programming languages on the basis of 
which functions they can be ..^^1 lo compute. However, Hewitt [b] has been able to compare 
classes of schemata, and to show, tor pxample. that in some well-defined senses schemata that 
permit recursive function calls up nm^ powerful than ones which forbid them It is this type of 
comparative study that should U tr. t.il for computational linguistics. 

The problem of findiiiii u.tiut .1 constraints for transformations faces the same difficulties as 
attempts to compare directly thr |m.w. , of programming languages. Instead of a direct analysis, we 
considered classes of schemr^ta t..i transformations, and tried to characterize what classes are 
reauired to give the computatK.n.Hl |.ower needed to analyze natural language by computer. It is 
important to have a clear idea of the computational power needed to analyze natural language. 



(12) See [7] and [5]. 
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6.2 The Analysis of Semantic Functions in Terms of Schemata 

As an example of how semantic transformations can be analyzed in terms of program 
schemata, consider again the sem.intic parse of the question 

Is 2 or } oddl 



Figure 2 (in Section 4 abov. ) presents a version of the surface syntax structure for this 
sentence, and Figure 3 gives the coi responding semantic deep structure. 

The schematological repi.^sontation for the computation needed to evaluate this is as follows 
Let xl be the list of choices, lot \2 be the set of odd numbers, and the interpretations of the 
functions be as follows: 



SI checks for an empty list (NULL) 
S2 -nppfMvis two lists together (APPEND) 
S3 is the subeet f-inction which accepts tuo arguments A and B 
and returns A if A i r. .I subset of B and returns NIL otherwise 
S4 select's the first element of a list (CAR) 
S5 selects the rest of a list (COR), 



The constant NIL is used to ipprcsent the empty list. The schema that represents the desired 
computation ts: 

(V k1 x2) - (repeat (z <- NIL) 
(if (SI A) then (return z) ) 
(r (S2 2 (S3 (S4 xl) x2))) 
(k1 <- (S5 xl))). 

This means that i is to be initiali/rd to NIL. and the remaining statements are to be executed until 
the condition in the first is true, thdt is. the list x I has become empty, and the (return z) statement 
is executed. The left arrow repie.^cnts assignment. The second statement builds the answer list (z) 
while the third reduces the argument list by removing from it the choice just considered in the 
second statement. 



6.3 Classes of Schemata 

Following Hewitt [5], it is possible to define a number of different classes of schemata. At 
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present we have investigateil only :wo such classes one that represents the non-transformational 
semantic functions of the CONsTRlJCT system, and one that represents all of the semantic 
functions that are used in the <\<^U'v) Althoujih we shall call the latter class the transformational 
schemata, it mcludes n i ^uImIu^ fhe non-tiansformational schemata. Intuitively, a non- 
transformational schema is mr f iv ^ ?o(]uei)ce ot assignment statements while a transformational 
schema is a non-recursive |ito-! nn ^ hpmi in the sense of Hewrt [5]. As should be obvious, there 
IS a transformational schrnvi Mm' r not equivalent under Hewitt's notion of equivalence to any 
non-transfoimational sch'^O) * ih« ot rhis are given in Rawson [15] 

Linguistically, a non ti »iri!iM m ioimI schema lepresnits a program for computing the value 
of 1 semantic parse by simply ium m MKi-order traversal ot the semantic parse tree. That is. one 
merely evaluates the nod^*s in uulri ^:c5|^[l^g with the innermost expressions and working outward 
This IS essentially the algorithin und by the top level of the LISP interpreter to evaluate 
expressions On the other f»an L • m Histoinaiional schema, which permits looping, allows one to 
change the order in whKh the ^u^- *Apie^sions of a semantic parse are evaluated, and permits the 
repeated evaluation of ceitain |)ui'i'tns ol the parse This seems to be the key computational 
concept in our notion of semantic m ^informations 



0 4 The Problem with Schcinaiology 

The major open probhin wiili our analysis of ?he computational structures needed to do 
semantic analysis is that the di^s rir i: ^n^form=^tional schemata \% too powerful: there are too many 
control structures which fall into rhp rla5S We must find some class intermediate between the non- 
transformational and the tranMui m ujonal schemata that icpresents more accurately the control 
structures really needed for natiii -tl l.^ngiiage undfrsiandin^ All that we can really claim to have 
done IS to have refntmuhf^d rh^ classical problem of finding natural constraints on 
transformations in terms which n>ikr it moic susceptible to solution. 



7 Conclusion 

The 700-rule grammar of the CONSTRUCT system recognizes a reasonable fragment of the 
sentences one finds in elementary mathematics texts Our choice of fragment emphasized 
complexity of structure over a large vocabulary, and hence we obtained a good variety of ways of 
asking the same questions. (13) 

(13) A modified version of CONSTRUCT is being used in a system for teaching set theory at 
Stanford beginning this fall 
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The original contnbuiion ot CONSTRUCT «s not the grammar itself or the use of set- 
theoretical structures as the semantic world for the meanmgs of utterances. The importance of the 
system is the notion of semantic tx'.nsjotmation. The important aspects of this notion are: 

1) The analysis of the surface syntax of an utterance is the control 
structure of a program that computes the meaning of the utterance. 

2) The execution of this program to compute the meaning of the 
utterance may involve functions that change the flow of cor rol, setting up an 
alternative control structuie. which we call the semantic deep structure. 

'i) T**f>'niqucs from the mathematical theory of computation, in 
particular schem itolot;y. may be useful in giving a formal characterization of 
these seman;.c transformations in a way that will both provide useful tools to 
the computational linguist and aid the general linguist as well. 
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